test(pyamber): add unit tests for attribute_type_utils by Amer-Mukhtar · Pull Request #6833 · apache/texera

Amer-Mukhtar · 2026-07-23T10:13:35Z

Description

What changes were proposed in this PR?

Add amber/src/test/python/core/models/schema/test_attribute_type_utils.py, a new pytest spec for the Python worker's attribute_type_utils, which previously had no dedicated unit tests.
Cover detect_attribute_type_from_arrow_field: every supported Arrow type, timestamps across all four units, the texera_type: LARGE_BINARY metadata marker taking precedence over the Arrow type, metadata that doesn't mark large binary falling back to the type mapping, and unmapped Arrow types raising KeyError.
Cover create_arrow_field_with_metadata: every AttributeType, metadata attached only for LARGE_BINARY, attribute names preserved verbatim (empty, unicode, emoji, long), and non-AttributeType input raising KeyError.
Cover round trips, including that stripping the metadata off a LARGE_BINARY field degrades it to STRING.

Any related issues, documentation, discussions?

Closes: #6635

How was this PR tested?

Ran: cd amber && pytest src/test/python/core/models/schema/test_attribute_type_utils.py -v, expect all 67 tests passing.
Test-only change; no production code is modified.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude

github-actions · 2026-07-23T10:14:45Z

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

No candidates found from git blame history.

github-actions · 2026-07-23T10:29:15Z

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 3 worse · ⚪ 12 noise (<±5%) · 0 without baseline

Compared against main 57d4dba benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	379	0.231	25,487/36,972/36,972 us	🔴 +8.0% / 🔴 +122.8%
⚪	bs=100 sw=10 sl=64	803	0.49	122,477/144,274/144,274 us	⚪ within ±5% / 🔴 +32.2%
⚪	bs=1000 sw=10 sl=64	901	0.55	1,109,136/1,147,208/1,147,208 us	⚪ within ±5% / 🔴 +10.7%

Baseline details

Latest main 57d4dba from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	379 tuples/sec	399 tuples/sec	754.55 tuples/sec	-5.0%	-49.8%
bs=10 sw=10 sl=64	MB/s	0.231 MB/s	0.243 MB/s	0.461 MB/s	-4.9%	-49.8%
bs=10 sw=10 sl=64	p50	25,487 us	24,980 us	12,816 us	+2.0%	+98.9%
bs=10 sw=10 sl=64	p95	36,972 us	34,240 us	16,594 us	+8.0%	+122.8%
bs=10 sw=10 sl=64	p99	36,972 us	34,240 us	19,806 us	+8.0%	+86.7%
bs=100 sw=10 sl=64	throughput	803 tuples/sec	808 tuples/sec	969.38 tuples/sec	-0.6%	-17.2%
bs=100 sw=10 sl=64	MB/s	0.49 MB/s	0.493 MB/s	0.592 MB/s	-0.6%	-17.2%
bs=100 sw=10 sl=64	p50	122,477 us	121,264 us	103,584 us	+1.0%	+18.2%
bs=100 sw=10 sl=64	p95	144,274 us	139,983 us	109,097 us	+3.1%	+32.2%
bs=100 sw=10 sl=64	p99	144,274 us	139,983 us	117,304 us	+3.1%	+23.0%
bs=1000 sw=10 sl=64	throughput	901 tuples/sec	900 tuples/sec	1,004 tuples/sec	+0.1%	-10.2%
bs=1000 sw=10 sl=64	MB/s	0.55 MB/s	0.549 MB/s	0.613 MB/s	+0.2%	-10.2%
bs=1000 sw=10 sl=64	p50	1,109,136 us	1,114,641 us	1,002,357 us	-0.5%	+10.7%
bs=1000 sw=10 sl=64	p95	1,147,208 us	1,172,225 us	1,046,463 us	-2.1%	+9.6%
bs=1000 sw=10 sl=64	p99	1,147,208 us	1,172,225 us	1,073,661 us	-2.1%	+6.9%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,527.89,200,128000,379,0.231,25487.09,36972.41,36972.41
1,100,10,64,20,2491.91,2000,1280000,803,0.490,122476.70,144274.12,144274.12
2,1000,10,64,20,22188.40,20000,12800000,901,0.550,1109135.77,1147207.89,1147207.89

codecov-commenter · 2026-07-23T10:36:55Z

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
46	1	45	1

View the top 1 failed test(s) by shortest run time

::src.test.python.core.models.schema.test_attribute_type_utils

Stack Traces | 0s run time

.../hostedtoolcache/Python/3.11.15................../x64/lib/python3.11.../site-packages/_pytest/python.py:507: in importtestmodule
    mod = import_path(
.../hostedtoolcache/Python/3.11.15................../x64/lib/python3.11....../site-packages/_pytest/pathlib.py:561: in import_path
    mod = _import_module_using_spec(
.../hostedtoolcache/Python/3.11.15................../x64/lib/python3.11....../site-packages/_pytest/pathlib.py:725: in _import_module_using_spec
    spec.loader.exec_module(mod)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.../hostedtoolcache/Python/3.11.15................../x64/lib/python3.11.../_pytest/assertion/rewrite.py:188: in exec_module
    source_stat, co = _rewrite_test(fn, self.config)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.../hostedtoolcache/Python/3.11.15................../x64/lib/python3.11.../_pytest/assertion/rewrite.py:357: in _rewrite_test
    tree = ast.parse(source, filename=strfn)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.../hostedtoolcache/Python/3.11.15................../x64/lib/python3.11/ast.py:50: in parse
    return compile(source, filename, mode, flags,
E     File ".../models/schema/test_attribute_type_utils.py", line 189
E       def test_non_attribute_type_raises_type_or_key_error(self, invalid_type):
E   IndentationError: unexpected unindent

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

Copilot

Pull request overview

Adds missing unit-test coverage for the Python worker’s core.models.schema.attribute_type_utils conversion helpers, ensuring Arrow ↔ AttributeType mapping (including LARGE_BINARY’s metadata-based encoding) is validated and protected against regressions.

Changes:

Introduces a new pytest module covering detect_attribute_type_from_arrow_field across supported Arrow types, timestamp units, metadata precedence, and error behavior for unsupported types.
Adds coverage for create_arrow_field_with_metadata, including LARGE_BINARY marker metadata behavior and round-trip preservation through Arrow schemas.

Comments suppressed due to low confidence (1)

amber/src/test/python/core/models/schema/test_attribute_type_utils.py:179

Similar to the earlier metadata check: comparing field.metadata to a plain dict can be fragile. Prefer asserting the presence/value of the texera_type marker to avoid version-specific equality behavior.

    def test_unicode_name_survives_on_a_large_binary_field(self):
        field = create_arrow_field_with_metadata("列名", AttributeType.LARGE_BINARY)
        assert field.name == "列名"
        assert field.metadata == LARGE_BINARY_METADATA

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Amer Mukhtar <136252708+Amer-Mukhtar@users.noreply.github.com>

test(pyamber): add unit tests for attribute_type_utils

72392be

github-actions Bot assigned Amer-Mukhtar Jul 23, 2026

github-actions Bot added the pyamber label Jul 23, 2026

Amer-Mukhtar marked this pull request as ready for review July 23, 2026 13:00

aglinxinyuan requested a review from Copilot July 24, 2026 03:45

Copilot started reviewing on behalf of aglinxinyuan July 24, 2026 03:46 View session

Copilot AI reviewed Jul 24, 2026

View reviewed changes

Comment thread amber/src/test/python/core/models/schema/test_attribute_type_utils.py

Comment thread amber/src/test/python/core/models/schema/test_attribute_type_utils.py Outdated

Amer-Mukhtar and others added 3 commits July 24, 2026 20:18

Potential fix for pull request finding

b21cc87

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Amer Mukhtar <136252708+Amer-Mukhtar@users.noreply.github.com>

Merge branch 'main' into test/attribute-type-utils

1767fd1

Merge branch 'main' into test/attribute-type-utils

1b0e12f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test(pyamber): add unit tests for attribute_type_utils - #6833

test(pyamber): add unit tests for attribute_type_utils#6833
Amer-Mukhtar wants to merge 4 commits into
apache:mainfrom
Amer-Mukhtar:test/attribute-type-utils

Amer-Mukhtar commented Jul 23, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 23, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Jul 23, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Amer-Mukhtar commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 23, 2026

Automated Reviewer Suggestions

Uh oh!

github-actions Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Benchmark changes need a look

Uh oh!

codecov-commenter commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 1 Tests Failed:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Amer-Mukhtar commented Jul 23, 2026 •

edited

Loading

github-actions Bot commented Jul 23, 2026 •

edited

Loading

codecov-commenter commented Jul 23, 2026 •

edited

Loading